Urban road disease intelligent diagnosis method fused with AI model and related equipment

Abstract

The invention provides an urban road disease intelligent diagnosis method fused with an AI model and related equipment, and relates to the technical field of road disease diagnosis. The method comprises the steps that automatic disease detection of the well lid is achieved through an intelligent diagnosis robot, specifically, the robot runs according to an automatic patrol route, after the robot reaches preset detection points of the well lid, the well lid is rolled three times along a Z-shaped route, and the route is designed based on the preset detection points at the two ends of the well lid. In the rolling process, a camera and a depth camera are synchronously used for collecting a well lid surface image and three-dimensional point cloud data of the well lid and a surrounding road surface correspondingly, and robot vibration data are recorded. And finally, analyzing the three-dimensional point cloud data and the vibration data by using a preset AI model, determining a well lid disease condition, and generating a disease report including a disease type and a corresponding well lid number. By implementing the method, multi-dimensional detection of well lid diseases is realized, surface visible diseases are included, and internal structure problems such as loosening and sinking can also be identified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road disease diagnosis, in particular to a city road disease intelligent diagnosis method fusing an AI model and related equipment. BACKGROUND

[0002] With the acceleration of urbanization process, the scale of urban road infrastructure is expanding, and the safety condition of road manhole covers, as an important part of urban underground pipe network system, is directly related to the safety of citizens' travel and the efficiency of urban operation. Timely and accurate detection of manhole cover diseases is of great significance to ensure the safety of urban road traffic and reduce maintenance costs. Currently, the detection of city road manhole cover diseases mostly relies on manual inspection. The inspectors check the damage, settlement, displacement and other diseases of the manhole cover through visual observation and tool measurement. A few areas have begun to use unmanned aerial vehicles to take pictures of the manhole cover and use deep learning models to identify diseases, which greatly improves the detection efficiency of manhole cover diseases.

[0003] However, the existing technology still has deficiencies. The existing image recognition method is greatly affected by environmental factors such as weather and light, and cannot effectively identify internal structural defects of the manhole cover and potential safety hazards caused by non-standard installation, which is prone to missed detection. SUMMARY

[0004] The present application provides a city road disease intelligent diagnosis method fusing an AI model and related equipment, which is used for efficiently and accurately identifying various diseases of city road manhole covers and surrounding pavements, and realizing automatic detection of city road diseases.

[0005] In the first aspect, the present application provides a city road disease intelligent diagnosis method fusing an AI model, which comprises: controlling an intelligent diagnosis robot to travel according to an automatic patrol route; when the intelligent diagnosis robot travels to a preset detection point of a manhole cover, controlling the intelligent diagnosis robot to perform three times of rolling on the manhole cover according to a Z-shaped route, the preset detection point comprising a first preset detection point and a second preset detection point at both ends of the manhole cover, a line connecting the preset detection points passing through a center point of the manhole cover and being consistent with the direction of the road; during the three times of rolling, synchronously capturing images of the surface of the manhole cover through a camera and acquiring three-dimensional point cloud data of the manhole cover and the pavement within a set range through a depth camera; when the intelligent diagnosis robot rolls over the manhole cover, synchronously recording vibration data of the intelligent diagnosis robot at this time, the vibration data comprising vibration frequency and amplitude data; based on the three-dimensional point cloud data and the vibration data, determining the disease condition of the manhole cover through a preset AI model, and generating a manhole cover disease report, the manhole cover disease report comprising the type of the manhole cover disease and the corresponding manhole cover number.

[0006] By adopting the technical scheme, the accuracy and comprehensiveness of disease identification are significantly improved through three different path rolling detection of the manhole cover, combined with multi-sensor data acquisition and AI model analysis. Specifically, the Z-shaped rolling path covers different stress points and detection angles of the manhole cover, the surface image captured by the camera can identify external defects such as cracks and peeling, the three-dimensional point cloud data obtained by the depth camera can analyze the structural characteristics of the manhole cover, and the vibration data reflects the combination state of the manhole cover and the road surface. After multi-source data is input into the AI model, multi-dimensional detection of manhole cover diseases is realized through fusion analysis, including not only visible surface diseases but also internal structural problems such as loosening and sinking, effectively reducing the limitations of single detection means and improving the reliability of disease diagnosis.

[0007] In combination with some embodiments of the first aspect, in some embodiments, the step of driving the intelligent diagnostic robot according to the automatic patrol route comprises: obtaining positioning information and number data of all manhole covers in the detection area; obtaining map data of the detection area; dividing the map data into multiple roads; selecting a road with the most manhole covers as the starting patrol road; after completing the patrol of the current road each time, selecting the next road that is connected to the current road and contains the most manhole covers; repeating the above steps until all roads are traversed.

[0008] By adopting the technical scheme, the road with the most manhole covers is preferentially selected as the starting point, ensuring that more detection targets are covered within a unit patrol distance, and reducing the invalid movement of the robot. In bidirectional road detection, a round-trip detection strategy is adopted, in which the current side goes from the starting point to the ending point, and the opposite side goes from the ending point to the starting point, avoiding repeated paths and turning operations, and shortening the overall patrol time. This path planning method not only improves the detection efficiency, but also reduces the energy consumption of the robot, prolongs the equipment endurance time, and is especially suitable for long-term patrol tasks of large-scale urban roads.

[0009] In combination with some embodiments of the first aspect, in some embodiments, after the step of selecting a road with the most manhole covers as the starting patrol road, the specific method of patrolling the road comprises: if the target road is a bidirectional road, dividing the target road into a current side road and an opposite side road; controlling the intelligent diagnostic robot to detect the manhole covers on the current side road in order from the starting point to the ending point; and when all the manhole covers on the current side road are detected, controlling the intelligent diagnostic robot to enter the opposite side road and drive from the ending point to the starting point to detect the manhole covers on the opposite side road.

[0010] By adopting the technical scheme, the bidirectional road is divided into the current side and the opposite side, so that the robot can concentrate on detecting the manhole cover on one side of the road, and the safety risk and time loss caused by cross-lane operation are reduced. The round-trip detection path from the starting point to the ending point and then from the ending point to the starting point avoids the problem of frequent U-turns or turns in the traditional one-way detection mode, and shortens the moving distance between adjacent manhole covers. This detection mode effectively improves the detection speed of a single patrol while ensuring comprehensive coverage, and reduces the impact on urban traffic.

[0011] In combination with some embodiments of the first aspect, in some embodiments, when the intelligent diagnostic robot drives to the preset detection point of the manhole cover, the intelligent diagnostic robot is controlled to perform three times of rolling on the manhole cover in a Z-shaped route, including: determining a center point of the manhole cover, and controlling the intelligent diagnostic robot to drive to the first preset detection point to obtain a reference line between the center point and the first preset detection point; controlling the intelligent diagnostic robot to perform straight-line driving in a direction at a first set angle with the reference line, to roll from one side edge of the manhole cover to the opposite side edge, to complete the first rolling; after the first rolling is completed, the intelligent diagnostic robot is controlled to perform turning to make the intelligent diagnostic robot face the center point, and the second rolling is performed; after the second rolling is completed, the intelligent diagnostic robot is controlled to perform turning to make the intelligent diagnostic robot face the second preset detection point, and the third rolling is performed.

[0012] By adopting the technical scheme, the first rolling is performed in a direction at a first set angle with the reference line, covering the diagonal line area of the manhole cover, and being able to detect cracks and other diseases distributed along the diagonal line; the second rolling is performed to face the center point, focusing on detecting defects in the central area of the manhole cover; and the third rolling is performed to face the second preset detection point, focusing on problems at the junction of the edge of the manhole cover and the road surface. This multi-angle rolling mode combines data collection of the camera, the depth camera and the vibration sensor, so that the AI model can analyze the state of the manhole cover from different perspectives, effectively identify the position, size and severity of various diseases, and provide a reliable basis for subsequent accurate repair.

[0013] In some embodiments of the first aspect, in some embodiments, the determining the manhole cover disease condition based on the three-dimensional point cloud data and the vibration data comprises: inputting each of the manhole cover surface images obtained by the three-time rolling into a manhole cover defect recognition model to obtain a plurality of manhole cover surface disease conditions, the manhole cover surface disease including cracks, peeling and potholes, the manhole cover defect recognition model being previously constructed by deep learning according to a plurality of manhole cover surface images with manhole cover defect annotations; and performing fusion processing on each of the manhole cover surface disease conditions, including: aligning images of the same region taken by the three-time rolling by an image registration algorithm; and determining whether a first region disease exists by a voting mechanism, specifically as follows: if at least two detection results of the detection results of the first region identify the same type of disease, it is confirmed that the first region has a disease, and a disease parameter is recorded, the disease parameter including crack length and width, peeling area and pothole depth; and inputting the disease parameter into a manhole cover surface disease classifier to obtain a manhole cover surface disease type and a severity level, the severity level being divided into mild, moderate and severe according to a preset threshold, the manhole cover surface disease classifier being previously constructed by deep learning according to a plurality of disease parameters with manhole cover surface disease type annotations and severity level annotations.

[0014] By using the above technical solution, the multiple detection results are fused by image registration and voting mechanism, reducing the misjudgment caused by factors such as light and shooting angle, and improving the reliability of disease recognition. Finally, the manhole cover surface disease classifier accurately determines the disease type and severity level according to the fused disease parameter. This hierarchical processing method not only ensures the comprehensiveness of detection, but also improves the accuracy of classification.

[0015] In some embodiments of the first aspect, after the step of inputting the disease parameter into the manhole cover surface disease classifier to obtain the manhole cover surface disease type and the severity level, the method further comprises: fusing each of the three-dimensional point cloud data obtained by the three-time rolling to construct a complete manhole cover road surface three-dimensional model; extracting manhole cover structure features from the three-dimensional model, the manhole cover structure features including manhole cover edge contour continuity and height difference between the manhole cover and the surrounding road surface; extracting vibration features from the vibration data to obtain a vibration feature vector, the vibration feature vector including main frequency distribution, amplitude attenuation characteristics and vibration duration; and inputting the manhole cover structure features and the vibration feature vector into a manhole cover structure disease prediction model to obtain a manhole cover structure disease type, the manhole cover structure disease type including manhole cover loosening, sinking and warping, the manhole cover structure disease prediction model being previously constructed by a random forest algorithm according to a plurality of manhole cover structure feature samples and vibration feature samples with manhole cover structure disease annotations.

[0016] By adopting the technical scheme, the manhole cover road surface three-dimensional model constructed by the three-dimensional point cloud data can extract structural features such as manhole cover edge contour continuity and height difference, and reflect physical form changes of the manhole cover; and the feature vectors of the vibration data reveal dynamic response characteristics of the manhole cover under stress. The two types of features are input into a prediction model trained by a random forest algorithm, so that structural diseases such as manhole cover loosening, sinking and warping can be effectively identified. These diseases are often difficult to directly observe on the surface, but will affect the service life of the manhole cover and driving safety.

[0017] In combination with some embodiments of the first aspect, in some embodiments, after the step of generating the manhole cover disease report, the method further comprises: in combination with the manhole cover number and the positioning information, marking the disease position on the electronic map to form a visual disease distribution heat map; based on the visual disease distribution heat map, obtaining a plurality of serious areas in which a heat value exceeds a set heat threshold; calculating a spatial density of the manhole covers in each of the serious areas; if the spatial density exceeds a set manhole cover density threshold, determining that the serious area has a spatial structure correlation; extracting the three-dimensional point cloud data of all the manhole covers in the serious area in which the spatial structure correlation exists; if the number of manhole covers with the same structural abnormality is greater than a set abnormality proportion, determining that there is a risk of underground structure associated disease in the serious area, and evaluating the risk of underground structure associated disease to obtain a potential risk evaluation; and pushing early warning information of the serious area to a municipal management department, the early warning information including a regional position, a disease density and the potential risk evaluation.

[0018] By adopting the technical scheme, the visual disease distribution heat map directly displays the spatial aggregation of the disease, and by setting the heat threshold and the manhole cover density threshold, the serious areas with dense diseases can be quickly identified. By analyzing the three-dimensional point cloud data of the manhole covers in these areas, the potential risk of underground structure associated disease can be found, such as the settlement of the foundation causing multiple manhole covers to simultaneously appear abnormal. The regional level risk evaluation can not only guide local repair, but also provide a decision basis for the overall planning of urban roads and the maintenance of underground facilities.

[0019] In the second aspect, the present application provides an intelligent diagnosis system, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors invoke the computer instructions to enable the intelligent diagnosis system to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0020] In the third aspect, the present application provides a computer readable storage medium comprising instructions that, when executed on an intelligent diagnosis system, cause the intelligent diagnosis system to perform the method described in the first aspect and any possible implementation manner of the first aspect.

[0021] In a fourth aspect, the present application provides a computer program product, which, when running on an intelligent diagnosis system, causes the intelligent diagnosis system to perform the method as described in the first aspect and any possible implementation manner of the first aspect.

[0022] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Since the multi-sensor fusion Z-shaped path rolling detection technology is adopted, the problem of incomplete detection and low accuracy of manhole cover disease in the prior art is effectively solved, and multi-dimensional accurate identification of surface disease and internal structure disease of the manhole cover is realized. Specifically, the Z-shaped path covers different stress points and detection angles of the manhole cover, the camera, the depth camera and the vibration sensor synchronously collect data, and the AI model analyzes the multi-source data, which not only identifies surface defects such as cracks and peeling, but also detects internal structure problems such as loosening and sinking, thereby significantly improving the reliability of disease diagnosis.

[0023] 2. Since the multi-angle Z-shaped rolling path planning technology is adopted, the problem of insufficient detection of local area of the manhole cover in the prior art is effectively solved, and omnidirectional and dead-angle-free disease detection of the manhole cover is realized. Specifically, the three times of rolling are carried out along different angles, the first time covers the diagonal area, the second time focuses on the central area, and the third time focuses on the edge joint, and combined with multi-sensor data collection, the structure and state information of each part of the manhole cover is comprehensively obtained, which provides rich analysis basis for the AI model, and ensures that various diseases can be effectively identified.

[0024] 3. Since the multi-model collaborative diagnosis technology of hierarchical fusion is adopted, the problems of low disease recognition accuracy and non-precise classification in the prior art are effectively solved, and high-precision identification and fine classification of manhole cover diseases are realized. Specifically, the manhole cover defect recognition model preliminarily analyzes the multiple rolling images, the image registration and voting mechanism fuse the detection results to reduce misjudgment, and the manhole cover surface disease classifier accurately judges the disease type and severity level according to the fusion parameters, and this hierarchical processing manner significantly improves the accuracy and reference value of the diagnosis result. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of the intelligent diagnosis method for urban road diseases in the embodiments of the present application, which fuses an AI model; Figure 2 is another flowchart of the intelligent diagnosis method for urban road diseases in the embodiments of the present application, which fuses an AI model; Figure 3 is an application scenario diagram of the intelligent diagnosis method for urban road diseases in the embodiments of the present application, which fuses an AI model; Figure 4 is a schematic diagram of an entity device structure of the intelligent diagnosis system in the embodiments of the present application. DETAILED DESCRIPTION

[0026] The terms used in the following embodiments of the present application are only for the purpose of describing the specific embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the present application, the singular forms "a," "an" and "the" are intended to include plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0027] Hereinafter, the terms "first", "second", "third", etc. are used only for the purpose of description and should not be understood as implying or indicating relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0028] For ease of understanding, the method provided by the present embodiment is described in the flow. Please refer to Figure 1 is a flowchart of the intelligent diagnosis method of urban road diseases in the embodiments of the present application.

[0029] S101, control the intelligent diagnosis robot to travel according to the automatic patrol route; Among them, the intelligent diagnosis robot represents a device with autonomous movement and manhole disease detection capability, usually integrated with camera, depth camera, vibration sensor and other detection devices, used for performing manhole disease inspection task on urban road. The automatic patrol route refers to the pre-planned robot driving path according to the manhole distribution and road communication of the area to be detected.

[0030] The timing of the step execution is after the intelligent diagnosis system starts the manhole cover inspection task. The scene is that the to-be-detected area contains multiple manhole covers that need to be detected for diseases. First, the system obtains the positioning information and number data of all manhole covers in the to-be-detected area. These data can come from the database of the municipal department or the previous inspection records. At the same time, the system also obtains the map data of the to-be-detected area. The map data can be a high-precision electronic map containing detailed information of the road. Then, the system divides the map data into multiple roads in order to plan the patrol route later. In order to improve the inspection efficiency, the system selects the road with the most manhole covers as the starting patrol road, so that more manhole covers can be detected in the early stage of patrol. After completing the patrol of the current road each time, the system selects the next road that is connected with the current road and contains the most manhole covers, and repeats this process until all roads are traversed.

[0031] The specific patrol method is that the system divides the target road into the current side road and the opposite side road according to the center line of the road or the traffic rules. For example, under the traffic rule of driving on the right, the right side road where the robot initially locates is the current side road, and the left side road is the opposite side road. Then, the system obtains the positioning information of all manhole covers on the current side road and sorts these manhole covers in the order from the starting point to the ending point of the road. The intelligent diagnosis robot is controlled to start from the starting point of the road and travel to the preset detection point of each manhole cover in turn. After the detection of all manhole covers on the current side road is completed, the system plans a path from the current position to the ending point of the opposite side road. This path may need the robot to turn at the intersection, U-turn, and other operations to realize. After the robot travels along the planned path to the ending point of the opposite side road, it starts to detect the manhole covers on this road. At this time, the detection order is from the ending point to the starting point of the opposite side road, which is opposite to the detection order of the current side road. Such a detection method can ensure that the robot does not need to return to the starting point again after completing the detection of one side road, but directly starts the detection from the ending point of the opposite side road, thereby improving the inspection efficiency.

[0032] S102, when the intelligent diagnosis robot travels to the preset detection point of the manhole cover, the intelligent diagnosis robot is controlled to perform three times of rolling on the manhole cover according to a zigzag route, the preset detection point includes a first preset detection point and a second preset detection point at both ends of the manhole cover, a line connecting the preset detection points passes through the center point of the manhole cover, and is consistent with the direction of the road; The first preset detection point and the second preset detection point are respectively located at two ends of the manhole cover, a line between the two points passes through a center point of the manhole cover, and is consistent with a road direction, and such a setting helps the robot to comprehensively detect the manhole cover from different angles. The Z-shaped route refers to a special driving path adopted by the robot when detecting the manhole cover, and the shape is similar to the letter "Z". Through this route, multiple and multi-angle rolling of the manhole cover can be realized, so that more comprehensive detection data can be obtained.

[0033] The specific implementation steps will be described in detail in subsequent steps S201 to S204, which will not be repeated here.

[0034] S103, during the three rolling processes, synchronously capturing the manhole cover surface image through the camera and acquiring the three-dimensional point cloud data of the manhole cover and the road surface within the set range through the depth camera; The camera refers to an image acquisition device installed on the intelligent diagnosis robot, which is used to capture two-dimensional images of the manhole cover surface. These images can reflect the crack, peeling and other disease conditions of the manhole cover surface. The depth camera is a device that can obtain three-dimensional information of an object. It calculates the distance of each point on the object surface by emitting laser or infrared light and analyzing the time or phase difference of the reflected light, thereby generating three-dimensional point cloud data. The three-dimensional point cloud data refers to a three-dimensional space data set composed of a large number of points, each point containing three-dimensional coordinate information, which is used to represent the three-dimensional shape and structure of the manhole cover and the surrounding road surface.

[0035] Specifically, the timing of this step is during the three rolling processes of the intelligent diagnosis robot on the manhole cover according to the Z-shaped route. The scene is that the robot needs to synchronously acquire the image and three-dimensional structure information of the manhole cover surface. During each rolling process, the camera will capture the manhole cover surface image at a certain frequency. The shooting frequency can be adjusted according to the driving speed of the robot and the detection accuracy requirement. At the same time, the depth camera also works synchronously to acquire the three-dimensional point cloud data of the manhole cover and the road surface within the set range. The set range is usually determined according to the size of the manhole cover and the possible disease influence range, and is generally the area within 0.5 meters to 1 meter around the manhole cover. In order to ensure the quality of the acquired data, the system will calibrate the camera and the depth camera to eliminate lens distortion and measurement error. During data acquisition, the system also records the time stamp of each shooting and data acquisition, so as to associate the image data and three-dimensional point cloud data with the corresponding rolling position and vibration data in the subsequent process.

[0036] S104, when the intelligent diagnosis robot rolls over the manhole cover, synchronously recording the vibration data of the intelligent diagnosis robot at this time, the vibration data including vibration frequency and amplitude data; The vibration data represents the mechanical vibration information generated by the intelligent diagnostic robot when rolling over the manhole cover, including vibration frequency and amplitude data. The vibration frequency refers to the speed of vibration, usually expressed in Hertz (Hz), and is used to reflect the structural characteristics and disease conditions of the manhole cover. The amplitude refers to the magnitude of the vibration, usually expressed in meters (m) or millimeters (mm), and is used to represent the intensity of the vibration.

[0037] Specifically, the timing of this step is when the wheels of the intelligent diagnostic robot contact and roll over the manhole cover. The scene is that the robot needs to obtain the vibration response information of the manhole cover under stress. During the process of the robot rolling over the manhole cover, the vibration sensors installed on the robot chassis or wheels will collect vibration data in real time. These vibration sensors can be accelerometers or gyroscopes, which can measure the vibration of the robot in three directions. The system will collect vibration data at a high sampling frequency (such as 1000 Hz or higher) to capture detailed information of the vibration. At the same time, the system will record the time stamp of the vibration data, so as to associate it with the corresponding manhole cover position, image data and three-dimensional point cloud data. In order to improve the accuracy of the data, the system will calibrate the vibration sensors and use filtering algorithms to remove noise interference.

[0038] S105, based on the three-dimensional point cloud data and the vibration data, determining the manhole cover disease condition through a pre-set AI model, and generating a manhole cover disease report, which includes the manhole cover disease type and the corresponding manhole cover number.

[0039] The system will input each manhole cover surface image obtained by three times of rolling into the manhole cover defect recognition model. The model has learned a large number of manhole cover surface disease characteristics of different types and different severity levels in the training stage, and can analyze the input image to identify whether there are cracks, peeling, potholes and other diseases in the image, and output the manhole cover surface disease condition obtained by each detection.

[0040] Due to the different driving angles and positions of the robot during the three times of rolling, the images taken may have angle deviation and position offset, so the manhole cover surface disease conditions need to be fused. First, align the images of the same area taken by three times of rolling through image registration algorithm, for example, by extracting feature points in the image (such as corner points of the manhole cover edge, end points of the crack, etc.), calculating the position transformation relationship of the feature points in different images, and then rotating, translating and other operations according to the relationship, so that the images of the same area can be accurately overlapped.

[0041] After aligning the images, a voting mechanism is used to determine whether a disease exists in the first region. For each region (i.e., the first region) on the surface of the manhole cover, the disease identification of the region in the three detection results is viewed. If at least two detection results identify the same type of disease, such as a crack in the first and second detection, it is confirmed that the region has that type of disease, and the corresponding disease parameters are recorded, such as the length of the crack is 5 cm, the width is 0.2 cm, etc.

[0042] After that, the recorded disease parameters are input into the manhole cover surface disease classifier. The classifier has learned the relationship between disease parameters and disease types and severity levels during training. The classifier will determine the type of disease (such as crack, peeling, pit) according to the input disease parameters, and determine the severity level according to the preset threshold. For example, for a crack, if the length is less than 3 cm and the width is less than 0.1 cm, it is determined to be mild; the length is between 3-10 cm and the width is between 0.1-0.3 cm, it is determined to be moderate; the length is greater than 10 cm and the width is greater than 0.3 cm, it is determined to be severe. Through this process, the type of disease on the surface of the manhole cover and the corresponding severity level are finally obtained. The manhole cover defect recognition model refers to a model based on deep learning technology for identifying diseases on the surface of the manhole cover, which needs to collect a large number of manhole cover surface disease samples, and label each sample in detail its disease type (such as crack, peeling, pit, etc.) and corresponding severity level (mild, moderate, severe). Then, use these labeled data to train the deep neural network, in the training stage, the deep neural network will constantly learn the internal relationship between disease parameters and disease types and severity levels. For example, when the length of the crack is longer and the width is wider, the corresponding severity level is usually higher. Through repeated learning of a large number of sample data, the neural network can automatically extract these complex features and rules, thereby realizing accurate classification and evaluation of unknown manhole diseases.

[0043] Crack represents linear damage on the surface of the manhole cover due to stress or aging; spalling refers to the flaky defects formed by the surface material of the manhole cover falling off; pit refers to the concave damage on the surface of the manhole cover. Image registration algorithm is an algorithm for aligning images of the same area taken at different angles and different times, ensuring that the corresponding areas in the images can be accurately matched. Voting mechanism is a method for integrating multiple detection results to improve accuracy, by counting the number of times the same type of disease appears in multiple detections, when a certain number of times, it is confirmed that the disease exists. Disease parameters are data that describe the specific characteristics of the disease, such as the length and width of the crack, the area of the spalling part, the depth of the pit, etc. The manhole cover surface disease classifier is a model based on deep learning for classifying and determining the severity level of the manhole cover surface disease, which divides the severity of the disease into mild, moderate and severe according to the preset threshold. According to the above embodiments, the surface disease of the manhole cover can be comprehensively diagnosed.

[0044] In some embodiments, after completing the analysis of the surface disease of the manhole cover, and having obtained the three-dimensional point cloud data and vibration data in the three rolling processes, the scene is to further determine the structural disease of the manhole cover. First, fuse each three-dimensional point cloud data obtained in the three rolling processes. Due to the different positions and angles of the robot during the three rolling processes, the three-dimensional point cloud data obtained may have some overlap and differences, and through the fusion algorithm (such as the iterative closest point algorithm), these data can be integrated together to construct a complete three-dimensional model of the manhole cover pavement, which can accurately reflect the overall shape of the manhole cover and the relative position relationship with the surrounding pavement.

[0045] Next, extract the structural features of the manhole cover from the constructed three-dimensional model. By analyzing the three-dimensional model, calculate the contour line of the edge of the manhole cover, judge its continuity, if the edge contour appears obvious fracture or misplacement, there may be structural problems; at the same time, calculate the average height difference between the surface of the manhole cover and the surrounding pavement within the set range, if the height difference is positive and large, it may indicate that the manhole cover is warped, if it is negative and large, it may be sunken.

[0046] Then, extract the features of the vibration data. The vibration data is a time series signal, and through Fourier transform, the frequency components can be obtained to determine the main frequency distribution; by analyzing the vibration amplitude curve with time, the amplitude attenuation characteristics can be obtained; record the time from the start to the end of the vibration, which is the vibration duration. These features together constitute the vibration feature vector, which reflects the vibration characteristics of the robot rolling over the manhole cover.

[0047] Finally, the extracted manhole cover structure features and vibration feature vectors are input into the manhole cover structure disease prediction model. In the training stage, a large number of sample data with manhole cover structure disease annotations (including manhole cover structure feature samples and vibration feature samples) are used, and the relationship between these features and disease types is learned through a random forest algorithm. The model analyzes and calculates the input features and outputs the manhole cover structure disease type, such as loosening, sinking, warping, etc., providing a basis for subsequent disease treatment.

[0048] Among them, the three-dimensional point cloud data fusion is the process of integrating multiple acquired three-dimensional point cloud data, removing redundant information, and filling in missing parts to construct a more complete and accurate three-dimensional model. The complete manhole cover pavement three-dimensional model is a model that reflects the three-dimensional form of the manhole cover and its surrounding pavement, which can clearly show the spatial structure of the manhole cover. The continuity of the edge contour of the manhole cover is a feature that describes whether the edge of the manhole cover is smooth and continuous. If the edge is broken or irregular, there may be a disease; the height difference between the manhole cover and the surrounding pavement refers to the vertical distance between the surface of the manhole cover and the surrounding pavement. An excessive height difference may indicate that the manhole cover is sinking or warping. The vibration feature vector is a vector extracted from the vibration data that can reflect the vibration characteristics; the main frequency distribution refers to the frequency distribution of energy concentration in the vibration signal; the amplitude attenuation characteristic refers to the speed of the vibration amplitude weakening over time; the vibration duration refers to the length of time from the occurrence to the disappearance of the vibration. The manhole cover structure disease prediction model is a model trained using a random forest algorithm to predict the type of manhole cover structure disease; manhole cover loosening refers to the loose connection between the manhole cover and the well seat, with shaking; sinking refers to the overall sinking of the manhole cover below the surrounding pavement; warping refers to the bending deformation of the manhole cover due to uneven stress.

[0049] In the embodiments of the present application, the detection path is dynamically optimized by the intelligent diagnosis robot according to the preset patrol strategy (such as preferentially selecting manhole cover dense roads and detecting on both sides of two-way roads), and the Z-shaped rolling mechanism is combined with multi-source data (images, three-dimensional point clouds, vibration data) for collection, and deep learning and machine learning models are used to analyze and fuse multi-dimensional features, so that the city road manhole cover detection area can be efficiently covered, avoiding missed detection and repeated detection, improving the comprehensiveness and accuracy of data collection, and through multi-level disease identification (surface defect classification, structure disease prediction) by the AI model, a three-dimensional diagnosis from the surface to the structure is realized, effectively solving the problems of low efficiency, high missed detection rate, and dependence on experience for disease identification and inability to quantitatively evaluate the structure risk in traditional manual inspection, thereby realizing intelligent, accurate and efficient diagnosis of city road manhole cover diseases, providing a data-driven scientific decision basis for municipal facility maintenance, and improving the intelligent level of city infrastructure management.

[0050] After combining the above scenarios, the method provided by the present embodiment is further described in more detail. Please refer toFigure 2 FIG. 6 is another flowchart of a city road disease intelligent diagnosis method using a fused AI model according to an embodiment of the present application.

[0051] S201, determining a center point of the manhole cover and controlling the intelligent diagnosis robot to travel to a first preset detection point to obtain a reference line between the center point and the first preset detection point; The center point of the manhole cover refers to the center position of the geometric shape of the manhole cover, which is usually the center of a circular manhole cover or the intersection of the diagonals of a square manhole cover, and is used to determine the symmetry reference of the detection path. The first preset detection point is a specific position at one end of the manhole cover, which is on the same straight line as the center point of the manhole cover and the direction of the connecting line is consistent with the direction of the road, and is used as the starting point of the third rolling. The reference line refers to the straight line connecting the center point of the manhole cover and the first preset detection point, which is used to determine the reference of the travel direction of the first rolling.

[0052] Specifically, the timing of this step is when the intelligent diagnosis robot arrives near the target manhole cover and completes positioning, and the scene is the initial path planning before the three-rolling detection of the manhole cover. First, the system obtains the three-dimensional point cloud data obtained by the depth camera or the image captured by the camera, and determines the center point coordinates of the manhole cover using geometric algorithms (such as circle fitting and rectangle diagonal calculation). For example, for a circular manhole cover, the center point is fitted after the contour is extracted by edge detection; for a square manhole cover, the intersection of the diagonals is calculated by detecting the four corner points.

[0053] After determining the center point, the system automatically generates the first preset detection point coordinates according to the preset rules (such as the end of the manhole cover along the forward direction of the road), and the connecting line between the two points needs to be parallel to the road direction (the road direction vector can be obtained through a high-precision map). Subsequently, the robot is controlled to travel along the current patrol route to the first preset detection point, and the path is corrected in real time through laser radar or visual navigation during the process to ensure the accuracy of the arrival (the error is usually controlled within ±5 centimeters). After arriving, the system records the coordinates of the center point and the first preset detection point, and calculates the direction vector of the connecting line as the reference line to provide a reference for the subsequent rolling direction.

[0054] S202, controlling the intelligent diagnosis robot to travel in a straight line in a direction with a first set angle with the reference line to roll from one side of the edge of the manhole cover to the opposite side edge to complete the first rolling; The first set angle refers to the angle between the first rolling driving direction and the reference line, which is usually set to 30-60° (e.g., 45°) to ensure that the rolling path covers the edge area of the manhole cover and forms a diagonal detection angle. Straight driving refers to keeping the robot in a constant direction without turning to ensure the straightness of the rolling track. The edge of the manhole cover to the opposite edge refers to driving from one edge of the manhole cover (e.g., the edge close to the starting point of the road) to the opposite edge (e.g., the edge far from the starting point of the road) along the set angle direction, covering the area in the diagonal direction of the manhole cover.

[0055] Specifically, the timing of this step is when the robot reaches the first preset detection point and completes the reference line calculation, and the scenario is to start the first rolling detection. The system calculates the target direction vector of the first rolling according to the reference line direction vector and the first set angle (e.g., 45°). For example, if the reference line direction is east, and the first set angle is 45° north, then the driving direction is northeast. Then, the robot is controlled to drive straight in this direction at a constant speed (e.g., 0.5 m / s), while the running distance and direction deviation are monitored in real time through the odometer and IMU. When the wheel touches the edge of the manhole cover (which can be triggered by a vibration sensor or visual edge detection), the data collection time point is recorded, and the first rolling is completed when the wheel leaves the opposite edge.

[0056] S203, after the first rolling, the intelligent diagnostic robot is controlled to turn to face the center point, and the second rolling is performed; The turning refers to the operation of adjusting the driving direction of the robot, which is achieved by driving the differential of the two sides of the wheel or the steering mechanism. Facing the center point means that the driving direction vector of the robot points to the center point of the manhole cover, so that the second rolling path passes through the center point, ensuring the key detection of the center area of the manhole cover. The second rolling refers to the rolling in a direction perpendicular to the reference line or passing through the center point, which is used to cover the area not involved in the first rolling and verify the center area defects.

[0057] Specifically, the timing of this step is after the first rolling is completed and the robot completely leaves the manhole cover area, and the scenario is to switch to the second rolling path. First, the system calculates the direction of the line connecting the current position of the robot and the center point of the manhole cover as the target direction of the second rolling. For example, if the first rolling direction is northeast and the center point is in the southwest direction of the current position of the robot, then the target direction is southwest (pointing to the center point). Then, the robot is controlled to adjust the driving direction through one or more turning actions (e.g., rotating in place or arc turning) until the center axis of the vehicle body is aligned with the center point.

[0058] During the turning process, the system identifies the feature points (such as the center point marker) on the manhole cover through visual sensors or uses laser radar point cloud matching positioning to ensure the accuracy of direction alignment (heading angle error ≤ 2°). After alignment, the robot is controlled to travel straight at the same speed along the target direction, from one side edge of the manhole cover to the other side edge, covering the path through the center point. The direction of the second rolling is usually perpendicular to the reference line (such as 45° for the first rolling and 135° or -45° for the second rolling), forming a cross-shaped detection path to ensure coverage of the main stress area of the manhole cover surface.

[0059] S204, after the second rolling is completed, the intelligent diagnostic robot is controlled to turn to face the second preset detection point, and the third rolling is performed.

[0060] The second preset detection point is a specific position on the other side of the edge of the manhole cover, symmetric about the center point of the manhole cover, and the line connecting the two points is consistent with the direction of the road, constituting an extension of the reference line. Facing the second preset detection point means that the direction of travel of the robot coincides with the line connecting the second preset detection point and the current position, and the third rolling path is symmetrically angled with the reference line, forming the last segment of the Z-shaped detection path.

[0061] Specifically, the timing of this step is after the second rolling is completed, and the robot is located at the edge of the manhole cover away from the first preset detection point (i.e., the end point of the second rolling). The scene is the completion of the last segment of the Z-shaped path rolling. The path of the second rolling is from one side edge of the manhole cover to the other side edge, pointing to the center point and extending in a direction perpendicular to the first rolling (such as 135°), so that the robot has traveled to the other side edge of the manhole cover (not the first preset detection point side) when it ends. At this time, the system needs to control the robot to turn so that the direction of travel is aligned with the second preset detection point, which is located at one end of the reference line away from the first preset detection point, and the distance from the center point is equal to the distance from the first preset detection point to the center point (such as both 0.5 meters).

[0062] During the turning process, the robot adjusts the direction through wheel speed difference or steering mechanism, combined with visual positioning (such as identifying the reference line marker on the manhole cover) or laser radar ranging, to ensure that the deviation of the direction of travel from the line connecting the second preset detection point does not exceed 3°. After turning is completed, the robot travels straight along the direction, from one side edge of the current manhole cover to the other side edge corresponding to the second preset detection point, forming a third rolling path. Since the first rolling is at a first set angle (such as 45°) with the reference line, the third rolling is at a negative first set angle (such as -45°) with the reference line, the two paths are symmetric about the reference line, and the second rolling is perpendicular to both, the three paths intersect to form a "Z" shape, which can cover the diagonal, positive, and negative diagonal areas of the manhole cover surface, avoiding the blind area of single direction detection (such as cracks in the corners of the manhole cover edge).

[0063] In the embodiments of the present application, since the reference line is constructed by accurately positioning the center point of the manhole cover and the preset detection point, and the Z-shaped path of three times of rolling is designed based on the reference line (the first time is obliquely covering the edge, the second time is vertically rolling the center, and the third time is symmetrically obliquely verifying the edge), combined with multi-direction data acquisition and path intelligent steering control, more than 95% of the area of the manhole cover surface can be covered from multiple perspectives, multi-dimensional detection data containing edge stress concentration area and center bearing area can be obtained, missing detection and feature misjudgment caused by single direction rolling can be avoided, the problems of incomplete coverage of traditional single path detection, high missing detection rate of edge disease, and single data dimension difficult to support accurate diagnosis are effectively solved, and then omnibearing scanning and fine identification of the surface disease of the manhole cover are realized, comprehensive and accurate basic data are provided for subsequent AI model fusion of multi-source data (image, point cloud, vibration) for disease classification and structure risk assessment, and the reliability and robustness of intelligent diagnosis are improved.

[0064] The scene of the present embodiment is supplemented as follows. Please refer to Figure 3 , Figure 3 which is a schematic diagram of an application scene of the intelligent diagnosis method of urban road diseases in the embodiments of the present application.

[0065] In Figure 3 (a), the preparation of the first rolling scene is shown, which includes a manhole cover (a circle), a center point (a center of the circle), a first preset detection point (a lower end point of the reference line), a second preset detection point (an upper end point of the reference line), an intelligent diagnosis robot (located at the first preset detection point), a reference line (a vertical line segment connecting the center point and the first preset detection point), and a first set angle (the angle between the reference line and the first rolling path). The intelligent diagnosis robot has traveled to the first preset detection point (the lower end point of the edge reference line of the manhole cover), recognized the manhole cover contour through a sensor (such as vision / laser radar), determined the center point (the center of the circle) after fitting the circle, and calculated the reference line (the vertical line segment in the figure) connecting the “center point-first preset detection point”. At this time, the system plans the first rolling path: the robot will roll obliquely from the current edge point to the opposite edge of the manhole cover at a first set angle (such as 45°) with the reference line (the oblique arrow in the figure), and completes the first oblique coverage detection.

[0066] In Figure 3 (b), the intelligent diagnosis robot (located at the right edge of the manhole cover, the second rolling end point), and the reference line (the vertical line segment connecting the center point and the first preset detection point) are included. After the first rolling is completed, the robot has traveled from the first preset detection point (the lower end point of the edge reference line of the manhole cover) to the second preset detection point (the upper end point of the edge reference line of the manhole cover), and the second rolling path is planned: the robot will roll vertically from the current edge point to the center of the manhole cover (the vertical arrow in the figure), and completes the second vertical coverage detection. Figure 3the end point of the oblique path in (a) in the first embodiment of the present application is the opposite side edge of the manhole cover, and the turning direction is adjusted to be opposite the center point (the path is a horizontal line segment, perpendicular to the reference line), and the second vertical rolling is completed from the right side edge of the manhole cover to the left side edge (or vice versa, depending on the actual direction). This path strictly passes through the center point, and focuses on detecting diseases (such as cracks and settlement) in the center area of the manhole cover, forming a "cross" intermediate detection path, covering the center area not fully involved in the first oblique path.

[0067] In the first embodiment of the present application, Figure 3 In (c) in the first embodiment of the present application, an intelligent diagnosis robot (located at the left side edge of the manhole cover, the end point of the second rolling), a reference line (a vertical line segment connecting the center point and the first preset detection point), and a third rolling path (oblique to the second preset detection point) are included. After the second rolling is completed, the robot is located at the left side edge of the manhole cover (or the end point position of the second rolling), and the system controls it to turn to be opposite the second preset detection point (the end point on the reference line), and plans the third rolling path: from the current edge point to the opposite side edge where the second preset detection point is located, along an oblique line with a symmetric negative angle (such as -45°, opposite to the direction of the first angle). This path is symmetrical to the first rolling path, forming a "Z-shaped" coverage, ensuring that diseases in the corner and symmetrical area of the manhole cover are detected, and finally completing the rolling detection process of three directions and the whole area.

[0068] In the first embodiment of the present application, the reference line and the initial path are accurately constructed through Figure 3 In the first embodiment of the present application, the reference line and the initial path are accurately constructed through Figure 3 In the first embodiment of the present application, the reference line and the initial path are accurately constructed through Figure 3 In the first embodiment of the present application, the reference line and the initial path are accurately constructed through

[0069] In some embodiments, after generating the manhole cover disease report, the disease distribution law can be analyzed from the spatial dimension to identify potential regional structural risks. First, the system binds the number of each disease manhole cover with its positioning information (such as GPS coordinates), and marks it in the form of an icon on the electronic map (such as a red icon representing a severe disease), and generates a disease distribution heat map through the kernel density estimation (KDE) algorithm. For example, on a certain main road, if multiple manhole covers have cracks and the positions are concentrated, the corresponding area of the heat map will show a deep red color.

[0070] Next, the system screens out areas with thermal values exceeding a set thermal threshold as severe areas (e.g., areas with thermal values > 80) and calculates the density of manhole covers in each severe area (e.g., the number of manhole covers per square kilometer). If the density of a certain area exceeds the manhole cover density threshold (e.g., > 20 per square kilometer), it is determined that the manhole cover disease in that area has spatial clustering, which may be associated with underground structures (such as waterway networks, roadbed settlement zones).

[0071] Then, the system extracts the three-dimensional point cloud data of all manhole covers in these areas and analyzes their structural features (such as edge contour breaks, height differences > 5mm). If the proportion of manhole covers with the same type of structural abnormalities (such as sinking) exceeds the set abnormality proportion (e.g., > 60%), it is inferred that the area may have a risk of associated underground structure disease (such as underground pipeline rupture leading to synchronous sinking of surrounding manhole covers). By further analyzing the spatial distribution pattern of three-dimensional point cloud data (such as abnormal manhole covers arranged along the direction of underground pipelines), combined with geological survey data, the risk level is evaluated (such as low, medium, high).

[0072] Finally, the system pushes warning information to the municipal management department, including: the geographic location of severe areas (e.g., the intersection of XX Road and XX Street), the density of disease (e.g., thermal value 95, spatial density 25 per square kilometer), and potential risk assessment (e.g., "underground pipeline leakage risk level: high, immediate investigation recommended"). The municipal department can start targeted maintenance according to the warning level, such as radar detection or excavation repair in high-risk areas. The manhole cover number is a unique identifier for each manhole cover, associated with its positioning information (such as latitude and longitude coordinates); the electronic map is a digital map of urban roads, supporting spatial location labeling and visual analysis; the disease distribution heat map is a two-dimensional graph that visually displays the concentration of disease by color depth, with darker colors (such as red) indicating higher disease density; the thermal threshold is the critical value that distinguishes ordinary areas from severe areas, which can be set according to historical data or industry standards (e.g., the top 10% of thermal values are defined as severe areas); the spatial density refers to the number of manhole covers or diseases per unit area, used to measure the clustering of diseases in the area; the associated risk of underground structure disease refers to the correlation risk of multiple manhole covers with the same type of structural abnormalities (such as sinking, warping) due to damage to underground pipelines, roadbeds, and other structures.

[0073] In the embodiments of the present application, since the disease heat map is constructed by the well lid number and positioning information, and after screening the serious area, the underground structure associated disease risk is analyzed layer by layer from the dimensions of spatial density, structural abnormality correlation and the like, the potential structural correlation of the disease aggregation area can be accurately mined, the limitation of single well lid detection is broken through, and the problem that the traditional detection cannot identify regional and associated underground diseases and leads to passive lag of municipal maintenance is effectively solved, and then active early warning, accurate evaluation and efficient pushing of the city well lid and associated underground structure risk are realized, and municipal management is upgraded from "single point repair" to "regional prevention and control".

[0074] The intelligent diagnosis system in the embodiments of the present application will be described from the perspective of hardware processing. Please refer to Figure 4 , which is a schematic diagram of an entity device structure of the intelligent diagnosis system in the embodiments of the present application.

[0075] It should be noted that Figure 4 The structure of the intelligent diagnosis system shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0076] As Figure 4 shown, the intelligent diagnosis system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 402 or programs loaded from a storage portion 408 to a random access memory (RAM) 403, such as performing the methods described in the above embodiments. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0077] The following components are connected to the I / O interface 405: an input section 406 including an audio input device, a push button switch, and the like; an output section 407 including a Liquid Crystal Display (LCD), and an audio output device, a lamp, and the like; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 409 performs a communication process via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as necessary. A removable recording medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is attached to the drive 410 as necessary so that a computer program read therefrom can be installed into the storage section 408 as necessary.

[0078] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present application. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer readable medium, the computer program containing a computer program for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 409, and / or installed from the removable recording medium 411. When the computer program is executed by the central processing unit (CPU) 401, various functions defined in the present application are performed.

[0079] Note that specific examples of the computer readable storage medium can include but are not limited to one or more of a conduit with one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0080] The flow diagrams and the block diagrams in the drawings are schematic and specific embodiments of possible architectures, functions, and operations of systems, methods and computer program products according to various embodiments of present application. It will be appreciated that each block in the flow diagrams and the block diagrams, and combinations of blocks in the flow diagrams and the block diagrams, can be implemented by various means, such as hardware, software, firmware, or any combination thereof. Also, the disclosure can be implemented by one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, the computer system 100, and according to a preferred embodiment, the computer system 100 can be a mobile device. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The computer readable medium can have stored thereon one or more computer program instructions which, when executed by the computer system 100, cause the computer system 100 to perform a method described herein. The computer program instructions can also be loaded onto the computer system 100 to cause one or more processors to produce a computer implemented process such as the process 200 described herein. As another aspect of the present application, an article of manufacture can comprise a computer readable medium having stored thereon, a computer program having multiple code sections. The computer program can comprise one or more code sections executable by a computer system to produce a computer implemented process such as the process 200 described herein. As a further aspect of the present application, a computer program product can comprise a computer readable medium having stored thereon, a computer program having multiple code sections. The computer program can comprise one or more code sections executable by a computer system to produce a computer implemented process such as the process 200 described herein. As yet another aspect of the present application, a computer system can be caused to perform the process 200 described herein by one or more computer program instructions stored on a computer readable medium. The computer program instructions can be executed by one or more processors of the computer system. Software or code segments can be downloaded to the computer system from the Internet and / or another computer system via, for example, a network or a communication link. Various implementations of the systems, methods and computer program products described herein can be realized. Therefore, these descriptions and representations are to be considered in all respects as only illustrative of the principles and concepts involved. Further, the scope of the application is indicated by the appended claims, rather than the foregoing description.

[0081] In particular, the intelligent diagnosis system of the embodiment includes a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, the intelligent diagnosis system implements the intelligent diagnosis method of urban road diseases based on the fusion AI model provided in the above embodiment.

[0082] As another aspect, the present application also provides a computer readable storage medium. The storage medium can be included in the intelligent diagnosis system described in the above embodiments, or can exist independently without being assembled into the intelligent diagnosis system. The storage medium carries one or more computer programs. When the one or more computer programs are executed by a processor of the intelligent diagnosis system, the intelligent diagnosis system implements the intelligent diagnosis method of urban road diseases based on the fusion AI model provided in the above embodiments.

[0083] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features. The modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0084] In the above embodiments, according to the context, the term "when" can be interpreted as meaning "if" or "after" or "in response to determining" or "in response to detecting". Similarly, according to the context, the phrase "on determining" or "if detecting (the stated condition or event)" can be interpreted as meaning "if determining" or "in response to determining" or "on detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)".

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned method embodiments can be implemented by a computer program instructing relevant hardware to complete, the program can be stored in a computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The aforementioned storage medium includes ROM or random storage memory RAM, magnetic disc or optical disc and various storage code medium.

Claims

1. A method for intelligent diagnosis of urban road diseases based on a fusion AI model, characterized in that, The method comprises: controlling the intelligent diagnostic robot to travel along an automatic patrol route; when the intelligent diagnostic robot travels to a preset detection point of the manhole cover, controlling the intelligent diagnostic robot to perform three times of rolling on the manhole cover along a Z-shaped route, the preset detection point comprising a first preset detection point and a second preset detection point at two ends of the manhole cover, a line between the preset detection points passing through a center point of the manhole cover and being consistent with a road direction; in the three times of rolling, synchronously capturing an image of a surface of the manhole cover through a camera and acquiring three-dimensional point cloud data of the manhole cover and a road surface within a set range through a depth camera; when the intelligent diagnostic robot rolls over the manhole cover, synchronously recording vibration data of the intelligent diagnostic robot at this time, the vibration data comprising vibration frequency and amplitude data; based on the three-dimensional point cloud data and the vibration data, determining a manhole cover disease condition through a preset AI model and generating a manhole cover disease report, the manhole cover disease report comprising a manhole cover disease type and a corresponding manhole cover number.

2. The method of claim 1, wherein, The step of controlling the intelligent diagnostic robot to travel along an automatic patrol route comprises: acquiring positioning information and number data of all manhole covers in a detection area; acquiring map data of the detection area; segmenting the map data into multiple roads; selecting a road with the largest number of manhole covers as a starting patrol road; after completing the patrol of the current road each time, selecting a next road connected with the current road and having the largest number of manhole covers; repeating the above steps until all roads are traversed.

3. The method of claim 2, wherein, The specific method of patrolling the road after the step of selecting a road with the largest number of manhole covers as a starting patrol road comprises: if the target road is a bidirectional road, dividing the target road into a current side road and an opposite side road; controlling the intelligent diagnostic robot to detect the manhole covers on the current side road in sequence according to an order from a starting point to an ending point of the road; after detecting all the manhole covers on the current side road, controlling the intelligent diagnostic robot to enter the opposite side road, travel from the ending point to the starting point of the opposite side road, and detect the manhole covers on the opposite side road.

4. The method of claim 1, wherein, The step of controlling the intelligent diagnostic robot to perform three times of rolling on the manhole cover along a Z-shaped route when the intelligent diagnostic robot travels to a preset detection point of the manhole cover comprises: determining a center point of the manhole cover and controlling the intelligent diagnostic robot to travel to the first preset detection point to acquire a reference line between the center point and the first preset detection point; controlling the intelligent diagnostic robot to travel in a straight line along a direction with a first set angle relative to the reference line, roll from one edge of the manhole cover to an opposite edge, and complete the first rolling; after the first rolling is completed, controlling the intelligent diagnostic robot to turn so that the intelligent diagnostic robot faces the center point and perform the second rolling; after the second rolling is completed, controlling the intelligent diagnostic robot to turn so that the intelligent diagnostic robot faces the second preset detection point and perform the third rolling.

5. The method of claim 1, wherein, The step of determining a manhole cover disease condition through a preset AI model based on the three-dimensional point cloud data and the vibration data comprises: inputting each of the manhole cover surface images obtained by the third rolling to a manhole cover defect recognition model to obtain a plurality of manhole cover surface disease conditions, the manhole cover surface diseases including cracks, peeling and pits, the manhole cover defect recognition model being constructed in advance according to a plurality of manhole cover surface images with manhole cover defect annotations through deep learning; fusing each of the manhole cover surface disease conditions, including: aligning the images of the same region taken by the third rolling through an image registration algorithm; determining whether the first region disease exists by using a voting mechanism, specifically as follows: if at least two detection results of the detection results of the first region recognize the same type of disease, it is confirmed that the first region has a disease, and the disease parameters are recorded, the disease parameters including crack length and width, peeling area and pit depth; inputting the disease parameters into a manhole cover surface disease classifier to obtain the type and severity level of the manhole cover surface disease, the severity level being divided into mild, moderate and severe according to a preset threshold, the manhole cover surface disease classifier being constructed in advance according to a plurality of disease parameters with manhole cover surface disease type annotations and severity level annotations through deep learning.

6. The method of claim 5, wherein, After the step of inputting the disease parameters into the manhole cover surface disease classifier to obtain the type and severity level of the manhole cover surface disease, the method further includes: fusing each of the three-dimensional point cloud data obtained by the third rolling to construct a complete manhole cover pavement three-dimensional model; extracting manhole cover structure features from the three-dimensional model, the manhole cover structure features including manhole cover edge contour continuity and height difference between the manhole cover and the surrounding pavement; extracting vibration features from the vibration data to obtain a vibration feature vector, the vibration feature vector including main frequency distribution, amplitude attenuation characteristics and vibration duration; inputting the manhole cover structure features and the vibration feature vector into a manhole cover structure disease prediction model to obtain a manhole cover structure disease type, the manhole cover structure disease type including manhole cover loosening, sinking and warping, the manhole cover structure disease prediction model being constructed in advance according to a plurality of manhole cover structure feature samples and vibration feature samples with manhole cover structure disease annotations through a random forest algorithm.

7. The method of claim 1, wherein, After the step of generating the manhole cover disease report, the method further includes: annotating the disease location on the electronic map to form a visual disease distribution heat map in combination with the manhole cover number and positioning information; obtaining a plurality of serious regions with heat values exceeding a set heat threshold based on the visual disease distribution heat map; calculating the spatial density of the manhole covers in each of the serious regions; if the spatial density exceeds a set manhole cover density threshold, it is determined that the serious region has a spatial structure correlation; extracting the three-dimensional point cloud data of all manhole covers in the serious region with the spatial structure correlation; if the number of manhole covers with the same structure anomaly is greater than a set anomaly proportion, it is determined that the serious region has a risk of underground structure associated disease, and the risk of underground structure associated disease is evaluated to obtain a potential risk assessment; pushing early warning information of the serious region to the municipal management department, the early warning information including the region location, disease density and the potential risk assessment.

8. An intelligent diagnostic system, characterized by, The intelligent diagnosis system comprises one or more processors and a memory; the memory is coupled with the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors invoke the computer instructions to enable the intelligent diagnosis system to perform the method in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, The instructions, when running on the intelligent diagnosis system, enable the intelligent diagnosis system to perform the method in any one of claims 1-7.

10. A computer program product, characterised in that, The computer program product, when running on the intelligent diagnosis system, enables the intelligent diagnosis system to perform the method in any one of claims 1-7.

Images